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ABSTRACT 

This study of the effectiveness of electronic 
blackboards in three introductory sociology classes at the University 
of Regina tested the hypothesis that this microcomputer-based 
presentational technology would have a negative impact on faculty 
productivity due to the increased amount of preparation time, and a 
positive impact on the quality of learning. Computer-prepared 
illustrated lectures were presented to two test classes, while a 
control class received the same material without the illustrations. 
Evaluation questionnaires using a five-point scale were administered 
prior to and after the experiment to all three classes. In addition, 
an in-class observer monitored all three classes, and all three were 
videotaped for review by a faculty panel. Follow-up interviews were 
conducted in both test classes. It was found that the electronic 
blackboard method was not successful because: (1) it took faculty 
five times longer to prepare an illustrated lecture than a normal 
one; (2) the electronic presentation interfered with normal patterns 
of classroom interaction; (3) students were dissatisfied with the 
electronic blackboard; (4) the electronic blackboard forced students 
to concentrate on two different sources of information 
simultaneously, resulting in a negation of any benefits gai.ied by 
superior organization and visual imagery; and (5) rural and low 
income students may have been negatively affected. It is concluded 
that, g.-i.ven the current state of the technology, the electronic 
blackboard is not effective under the conditions tested. Copies of 
the study consent form and the pre-experiment , control group, and 
test group questionnaires are appended* (5 references) (DB) 
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Executive Summary 

Evaluating the Electronic Blackboard 



Hypothesis 

An electronic blackboard is the use of microcomputer-based presentational 
technology to illustrate lectures. We expected that electronic blackboards would impact 
negatively on faculty productivity (a slight increase in the quality of teaching would not 
offset the large amount of time needed for preparation) and positively on the quality of 
learning. 

Procedures 

We conducted a controlled experiment involving three introductory sociology 
classes at the University of Regina in the fall 1989 semester. Computer-prepared 
illustrated lectures were presented to two test classes, while a control class received the 
same material without illustrations. Evaluation questionnaires using a five-point scale 
were administered prior to and after the experiment to all three classes. An in-class 
observer monitored all three classes, and all were videotaped for review by a faculty panel. 
Follow-up interviews were conducted in both test classes. 

Results 

The electronic blackboard was not successful in our experiment. We measured 
negative impacts on all of our variables, 

- Faculty productivity declined sharply. It took five times as long to prepare an 
illustrated lecture as a normal one. 

- Teaching quality declined as the electronic presentation interfered with normal 
patterns of classroom interaction. 

- The students were dissatisfied with the electronic blackboard. The evaluations of 
both test classes were not significantly different from the control group but were 
much worse than the pre-experiment classes, 

- The electronic blackboard forced students to concentrate on two different sources 
of information simultaneously. The resulting division of attention and distraction 
more than negated whatever benefits might have been gained by superior 
organization and visual imagery. 

- Gender was apparently not a major factor, 

- There is some (weak) evidence that rural and low income students were negatively 
affected by the electronic blackboard. 

Conclusion 

Given the current state of the technology, the electronic blackboard is not effective 
under the conditions tested. Some forms of computerized teaching may be effective in the 
university classroom, but we believe that at this time the electronic blackboard is not one of 
them. 



Over the past ten years microcomputers have become ubiquitous in North American 
universities. Ubiquitous, that is, except in the classroom. While some disciplines make 
extensive use of computers for teaching purposes, in a variety of ways, others apparently 
rarely, if ever, use computers in the classroom. At the same time, students' pre-university 
experiences have been shaped by these new technologies. Many have speculated that rock 
video, VCRs, video games, and the all-pervasive television are changing the way people 
perceive the world (cf.: McLuhan, 1964; Meyrowitz, 1985). Bold graphics, special 
effects, and iconography have become the hallmarks of what some claim is a more 
visually-oriented world. 

Canadian higher education relies extensively upon lecture format classes at the 
introductory level, especially in the social sciences and humanities. Can the graphics 
capabilities of microcomputers be used to enhance these clFsses? If our students are 
coming to university already oriented to visual images, it would seem that there would be 
considerable potential for an "electronic blackboard." 

Unfortunately, the literature pnvides few guidelines. While there is a vast literature 
on the use of microcomputers m teaching, little of it involves evaluation. Much is either of 
the "golly-gee-whiz, look what I did" variety or is written in the future tense- more 
concerned with what computers could do than evaluating actual performance (cf.: 
Danzinger, 1985). In one review of the literature, Jolicoeur and Berger (1986:7) found 
only 47 outcome evaluation studies and of these only two met the conditions of evaluating 
an individual software package with an objective test and a control group. Very little is 
published on electronic blackboards in university teaching. 
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Will the use of an electronic blackboard improve faculty productivity and the quality 
of learning? We tested this hypothesis in three introductory sociology classes at the 
University of Regina during the fall semester of 1989. U of R introductory sociology 
classes are fairly typical of those across Canada, They are medium to large sized (45 to 
well over 100 students) lecture format classes using a survey approach. Most of the 
students are in their first year and relatively few have a declared major in the social 
sciences. For the majority of students this is a service class used to fulfill university 
requirements. Two of our project team (Knuttila and Stahl) have a combined total of 36 
semesters experience teaching these classes. 

We define an electronic blackboard as microcomputer-based presentational 
technology. In this test we used More II presentational software, augmented by various 
graphics packages, on a Macintosh Ilex to generate colour iiiustrated lectures which were 
then displayed on a large screen. 

Our dependent variables were faculty productivity and quality of learning. We 
defined an increase in faculty productivity as an increase in uncommitted faculty time 
(which, ceteris paribus, should lead to increased research, publications, etc.) and in the 
quality of teaching. The quality of teaching is more subjective but can be judged by peer 
review. An increase in the quality of learning is defined as improved student satisfaction 
and performance. 

We expected that electronic blackboards would impact negatively on faculty 
productivity (a slight increase in the quality of teaching would not offset the large amount 
of time needed for preparation) and positively on the quality of learning. Whether this 
would result in a net gain or loss would be a judgment dependent upon our results, 

PROCEDURES 

If the electronic blackboard "works'' it should enhance those situations which 
students find most boring and which faculty find most difficult to teach. In our experience. 



the three most difficult areas in an introductory sociology class are history, statistics, and 
theory. Consequently, we prepared three illustrated lectuies: an historical profile of 
religion in Canada, a statistical profile of religion in Canada, and secularisation theory. 
The lectures were then given to three sections of Sociology 100 over a one week period. 
The two test classes saw the illustrations on a large screen while the control group had the 
same lectures without the electronic blackboard (an overhead projector was used for several 
maps and graphs in the control group). Initially, a second round of lectures was planned 
for one class which would have divided the group mto control and test subsections in order 
to measure student performance. Unfortunately, excessive preparation time (see below) 
forced the cancellation of this portion of the experiment. 

We used multiple measures for each of the variables. For faculty productivity we 
measured faculty time and the quality of teaching. In order to obtain a real test of 
preparation time we had to include the learning curve for both hardware and software. 
Therefore, one of us (Stahl), who was moderately experienced on microcomputers but was 
new to both the Macintosh and computer graphics, prepared all the lectures, keeping a log 
of his time. The first two lectures were somewhat "off the sheir and only needed revision 
and the addition of illustrations. The third was prepared "from scratch," and he held back 
from working on it until he was confident in his use of the machine and software. We thus 
obtained a measure of botn the learning curve and of preparation time for a somewhat 
experienced user. 

The quality of teaching is inherently a more subjective evaluation. Peer review, 
however, should give as accurate a measure of teaching as it does for proposals or 
publications. An in-class observer (Johnson) moti'tored all three classes prior to and 
during the experiment. One of the pre-experiment classes (Knuttila's day class) was 
videotaped on two occasions and all the control and test lectures were videotaped. A panel 
of faculty later reviewed the tapes for evaluation. 
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The chief indicator of the quality of learning was student satisfaction. Because the 
second round of the experiment had to be cancelled, we had no controlled means of 
obtaining data on ? tudent performance. Student satisfaction was measured through 
questionnaires and interviews. Questionnaires (see appendix B) were administered to all 
three classes several weeks prior to the experiment and again at its conclusion. The 
instrument recorded socio-economic and academic data, used a five-point attitudinal scale, 
and included several essay-type questions for additional opinion. The questionnaires for 
the two test classes differed slightly in the wording of several questions (to make them 
relevant to the presentation) and in having one additional question (which did not apply to 
the non-test situations) but were fully comparable. Following the experiment, Johnson 
interviewed five students from each test class. Each interview lasted between 30 and 45 
minutes and was recorded and transcribed under a pseudonym. 

DIFFICULTIES 

In the course of the experiment we encountered several practical and theoretical 
difficulties. The first was software. Because it is oriented so much towards business, the 
computer industry has not often taken the particular needs of the university into account. A 
business person can devote time and resources to a presentation because usually it will be 
used many times. A university professor needs many presentations each of which may be 
used once. While More II was not a particularly difficult program to learn and use, we 
wanted more than simple bullet charts so additional graphics had to be imported. This 
required both learning several graphics packages and purchasing collections of electronic 
"clip art" Since the paint and draw programs we had did not produce charts and graphs, a 
graphing program was also needed Each additional software package multiplied the cost 
as well as the learning and preparation time. Clip art packages are available in a variety of 
formats, but most are heavily oriented to business. Little is available which is usable in an 
historical or sociological presentation. In order to obtain enough illustrations we had to 
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locate pictures, capture them on a banner, and then edit them in a graphics program. Once 
again, learning and preparation time multiplied. 

Displaying the lectures on a large screen proved surprisingly difficult. A single 
monitor is too small to be seen by a large class. A "smart classroom" containing a number 
of networked monitors was not available to us at the University of Regina. LCD projector 
pallets which utilize an overhead projector do not use colour. We finally settled on an 
RGB projector and a large-screen TV, both of which requiied the use of an NCC 
converter. This seriously degraded the quality of the picture. Since we had to use the 
converter anyway, we videotaped the presentations and showed them using a four-head 
VCR with stop-frame control. 

Co-operation and co-ordination also proved difficult. Dr. Barry Anderson, whose 
morning class served as our control group, gave us splendid co-operation. Unfortunately, 
the University of Regina Audio- Visual Services did not. Their denial of equipment caused 
delays and forced us to move both test classes out of their usual classrooms into much less 
satisfactory venues. Even with full co-operation between Drs. Anderson, Knuttila, and 
Stahl, co-ordination was difficult. It proved impossible to "fit"" the presentations smoothly 
into the flow of the classes. Consequently, the control and test sessions were intrusive 
episodes. Tlie use of a guest lecturer for the presentations added the complicating factors 
of differences in personality and tesiching style. 

In addition to these practical difficulties, there were several theoretical problems 

which we anticipated and attempted to control. First of these was the selectivity problem. 

As Lieberson reminds us; 

A key feature of the experimental approach is that the subjects ... are randomly 
assigned to the conditions under study. If the assignment process is not random, 
then the investigator must be fully satisfied that it has no bearing on the likely 
outcome (1985:14). 

This affects the validity of the study- does it measure what it intended to measure? In our 
case, each class is unique and not fully comparable. The size of the class, the time of day it 
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meets, the configuration of the classroom, the age. sex, experience, and nr^tivation of the 
students are all factors which could affect the outcome but which could not be controlled. 
However, by collecting background socio-economic and academic data and by using 
multiple measures in pre, control, and test situations we tried to detect the significant 
factors affecting the outcome. 

Reactivity is also a problem. Our subjects were aware of the study and perhaps 
behaved differently because of it. For ethical reasons, the students were informed of the 
experiment early in each course and given a consent form (see appendix A). The guest 
lecturer, videocamera, presentational technology (and in the case of both test classes, 
moving to a different classroom) were intrusive. Several of our methods mitigated this 
problem. The in-class observer (Johnson) is a graduate student who easily and 
unobtrusively "fits in." Questionnaires were anonymous. All the interviews were 
conducted by Johnson, recorded under a pseudonym, and the interviewees were both 
given an honorarium and assured that neither their class teacher nor die guest lecturer 
would be able to identify them. The students of all three classes were co-operative and 
gave frank evaluations ("God help India" one student in a test class wrote). We expected 
that many would react to the novelty of die situation, rating the presentations higher simply 
because they were different. Instead, the written comments lead us to believe that a 
number of students responded negatively to changes in their usual routine. 

RESULTS 

The results were not what we had expected. Instead of a slight net loss in faculty 
productivity and a g^n in the quality of learning, we measured negative impacts on all 
variables 

Learning and preparation time were far greater than we had anticipated. The 
combined learning and preparation time for the first two lectures (which were developed 
simultaneously) totalled 63.25 hours. These were revisions of existing lectures, so none 
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of this time represents reseaich or planning. Figure 1 breaks the time down by software 
package. 'Scanner' represents time locating art and editing the captured images as well as 
using the scanner itself. 'Video' represents production time recording the presentations on 
videotape. 'Miscellaneous' is time spent using several programs simultaneously, usually 
transfering illustrations from graphics packages to More II. The relative time spent on each 
of these is shown in Figure II. The effects of having to learn and use multiple software 
packages, scanners, and clip art is readily apparent. 



FIGURE I 



Learning and Production Time 
First Two Lectures 
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FIGURE II 



Learning and Production Time 
First Two Lectures 
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FIGURE 

Preparation Time 
Third Lecture 
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With experience, preparation time declined dramatically but was still excessive. 
The third lecture, on secularisation theory, was new material although all the research was 
complete. It took four hours five minutes to plan and write the lecture. At this point it 
could have been presented normally. It took an additional 16.2 hours to illustrate the 
lecture, for a total of 20 hours 1 7 minutes (see Figure III). Thus for a moderately 
experienced user with research complete and all materials at hand, it tock five times as long 
to prq)are an illustrated lecture as the same material presented n^^rmally. 

In spite of the lengthy preparation time, Stahl and the faculty review panel (Drs. 
Brien Maguire and Barbara Kaltz) agreed that the quality of teaching suffered. There were 
several reasons for this. Coming in as a guest lecturer, Stahl felt he had little rapport with 
the students, yet his presence for a week was long enough to eliminate any novelty to the 
situation. This was confirmed by several students' comments, complaining of having to 
adjust to a different teacher's style. The lighting was not good for either test. Both 
classrooms had to be darkened to see the screen, and in the day class the lecturer was 
silhouetted by back-lighting. This impeded both faculty and students from establishing 
eye-contact. The use of illustrations focused the concentration of both student and lecturer 
on the screen, splitting the attention of the class. This became quite obvious in one of the 
videotapes as students' heads can be seen jerking up from their notes, to the professor, 
across to the screen, then back to their notes in a mechanical fashion. The normal 
classroom patterns of interaction were interrupted. Indeed, the entire electronic blackboard 
format proved to be non-interactive. As one student put it, "It was like watching a movie." 
If the illustrations did not carry the presentation, there was not enough human interaction to 
make up the difference. This had repercussions on the control class as well. Since the 
control class had the same lectures, sans illustrations, the rigid format similarly limited 
interaction, resulting in a dull presentation. 

Our biggest surprise was the strongly negative response from the students. In 
almost every category they expressed dissatisfaction with the electronic blackboard. 
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Table 1 shows the mean evaluation scores. Each column identifies the class, whether it 
was a pre, control, or test group, the date the evaluation was done (in Anderson's morning 
class evaluation was done in seminars which were spread over several days) and an 
identification code assigned to each class. Keywords identify each of the evaluative 
questions. Those marked (-) are worded so that a low number on the five-point scale 
indicates a favourable response. 

TABLE 1 
MEAN EVALUATION SCORES 





Morning 
Pre 


Morning 
Control 


Night 
Pre 


iNight 
test 


^Day 
Pre 


.Day 
Test 






11-13 Oct 


20-25 "Oct ; "Te-bct ' 23-Oct' ' 1 C 


^Octi 3 


-Nov 


KEYWORDS 


class 01 


class 04 class 


02 jclass 


05 ^class 


03 jclass 


06 


10 organize 


3.89 


"■'"""3.541 


3 911 


3.95' 


4.16' 


3.95 


11 compport 


3.62 


2.54 




~3.5l' 


2.85' 


3.71 i 


2.35 


12 effectiv 


3.85 


2.52 




3.76: 


2.97! 


4.08" 


2.48 


13 appropri 


3.89^ 


3.18! 


3.89^ 


3; 48! ' 
2.90^ 


4.26i 


3.36 


1 4 compclas 


3.92 


2.45T 


4.641 


4.36 


2.58 


15 pace 


3.46 


2.63 




3T29! 


2.72 


3.55' 


2.51 


speed 


2.00 


" 1.45 




1.33; 


1.50' 


1 .40' 


1.77 


16 quality 


3.80 


2786 
~3.0"0' 




"3.93; 


3.03 


4.00 


2.65 


17 instruct 


4.18 




4.42; 


3.68* 


4.58^ 


3.10 




















18 boring(-) 


1.94 






"2. 02; 
l791^ 


3.10; 


1 .94 


3.20 


19 see (-) 


2.00 


2.80 




3.46' 


1. 72' " 


3.38 


20 notes 


3.26 


2.40 




"3;02i 


2.90' 


3.64* 


2.98 


21 _djfflcul 

22 inrerup(-) 

23 understa 


1.94 
2.90 
3.78 


1.96| 
2 75' 

3.01: 


2.06i 

2.62; 

3.81 


2.18' 
2.77' 
2.69 


1.84' 

2.78 

4.06' 


1.92 
2.98 
2.60 


24 remember 


3.59 


2.76 




"3.7^ 


2.69! 


3.94- 


2.63 


25 more 


3.61 


2.59 ■ 


3._63: 


2.77' 


3.76' 


2.50 


26 distract(-) 


n=52 


n=76 ;n=53 


n=40 


3.13' 

in =50 


*n=40 


3.40 
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In order to set up a rigourous test, we had chosen the topics of the lectures because 

it was material which students usually found boring and faculty found difficult to teach. 

As the scores for the control group indicate, we succeeded. Control equalled pre on only 

two questions; numbers 21 ("This material is too difficult") and 22 ("During the 

presentation, I am afraid to interrupt with a question"). Students overwhelmingly found it 

more boring. Overall, the control group averaged .85 worse than the pre group on the 

five-point scale. Several reasons can account for this. In part, the control class suffered 

by comparison. Dr. Anderson is a popular, effective teacher who had just completed a unit 

on socialization, a topic which, in our experience, is very popular with introductory 

students. In part the format of the lectures-- required for comparability with the test 

classes- forced the lecturer into a style which restricted interaction. The lectures were a 

"show down front" but without much of a show. But as the students' crmmented 

repeatedly, most of all it was the content of the lectures themselves which they did not like. 

The use of the electronic blackboard did nothing to improve students' satisfaction. 

The test classes equalled the pre scores on only three questions, numbers 10 ("The 

organization of the presentation was:") and questions 21 and 22 where the control group 

also did well. Overall, the night class test averaged .70 less favourable and the day class 

test a massive 1.10 less favourable than their respective pre-experiment classes. Neither 

test class was significantly different from the control group: the night class averaged .17 

better than control and the day class .01 worse. (The day test class also averaged .19 less 

favourable than the night class.) 

A look at the individual questions reveals some patterns. The place where the 

electronic blackboard was most successful was its ability to make a clearly organized 

presentation. The bullet charts laid out the lectures in a clear, easy-to-follow manner. A 

number of students expressed opinions similar to those of "Bob" from the night class: 

Again just the organization of it (made the presentation good]. It was nice to have 
definite subject headings. I have always had a problem with that. We didn't learn a 
lot about that in high school. We didn't even take notes on our own in high school 
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grade 12, so I always had difficulty in just organizing my subject headings and 
knowing what was relevant and that kind of helped. 

As Table 2 shows, on question 10 ("The organization of the presentation was:") the test 

classes outscored all others in the "excellent" category. It is the only question on which the 

electronic blackboard did so. The test classes were significantly superior to the control 

group on question 10, but were not significantly different from the pre-tests. 



TABLE 2 

FREQUENCY DISTRIBUTION PERCENTAGES 



10. The organization of this presentation was: 
20. The presentation helped me to take notes better: 
26. I find the computerized presentation distracting: 



CLASS 

10 ORGANIZE 

01 pre 

02 pre 

03 pre 
0 4 c p n tip l 

05 test 

06 test 



20. NOTES 

01 pre 

02 pre 

03 pre 

04 cpntrpi 

05 .tes~t 

06 test 



iPOOR 



FAIR 



A\/ERAGE 



QOCD 



EXCELLENT 



4- 



0 

o" 
o' 

0 
0 

o' 



0^ 
10.5! 

7.5' 
12.5! 



25 61.5 

22.6 58.5 

6^ 72 

34.2' 46.1 

12.5 57.5 

2.5 62.5' 



I ST DISAGREE DISAGREE 



,..i..„ 



t - 



3.8 
7. 5' 
0 

14.5' 
20' 
17.5 



13.5 
17 
22 
9.2 
22.5 
22.5 



jUNDEaOED AGRS 

13.5: 46.2 23.1 

22.6; 34 24.5' 

12 3 8 2 4 

39.5 36.8 6.6 

22.5- 12.5 32.5' 

20 17.5 37.5 



ST. AGREE 



11.5 
7.5 
26 
1 .3 
1 0 
7.5 



26. DISTRACT (-) 

05 test ' 

06 test ' 



1 0 
7.5 



27.5; 
25' 



20 
12.5 



20 
30 



20 
25 
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The reasons for this appear when we look at questions 20 ("The presentation helped 

me to take notes better'') and 26 ("I find the computerized presentation distracting''). The 

test classes were significantly superior to the control class on question 20 but inferior to the 

pre-experiments (although not significantly so in the night class). Organization helped 

some students take better notes, but organization alone is not the decisive factor. As 

"Betty'' from the day class explained: 

Well it depends, if you get a prof that just stands there and talks and he doesn't 
write anything on the board, it's a lot better because, you know, you can see on the 
screen and it helps you write better notes. But, if some profs write on the board all 
the time and they explain as they are going, I find it a lot better than using the 
screen. 

She adds: "Because he's talking away and you're reading what's on the screen and I just 

missed half of what he said." This divided attention was a recurring theme in the 

interviews. The electronic blackboard forced students to concentrate on two sources of 

information at once. For some students this was not a problem, but for large numbers it 

was a disaster (see Table 2; Figure IV). It is also a problem inherent in the electronic 

blackboard As "Chuck" from the night class explained; "Well, it was just one other 

element that you had to contend with as opposed to the listening and looking at the 

chalkboard." " Al" from the day class described the effect: 

Well, you are always watching the screen because you are afraid to miss a change 
or something* Because the change wasn't quite obvious and there was no sounds 
and you would have to watch for the changes so it kept your attention. Its not like 
where you are writing on a challd)oard you know you can hear the prof writing on 
the chalkboard and there is some other action involved. So you have to keep alert to 
make sure what is going on. 

The students' divided attention detracted from their ability to take notes and largely negated 

whatever benefits were gained by clear organization. 

This same problem appears in questions 23 ("The computerized presentation helped 

me to better understand the material") and 24 ("The computerized presentation helped me to 

better remember the material). Both questions are theoretically important. If, as some have 

speculated, students today are more visually oriented the electronic blackboard should be 
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FIGURE IV 

FREQUENCY PERCENTAGES 
26. I find the computerized presentation distracting: 




■ 05 test 
□ 06 test 



ST DISAGREE DISAGREE UNDEGDED AGREE ST. AGREE 



TABLE 3 

FREQUENCY DISTRIBUTION PERCENTAGES 

23. The computerized presentation helped me to better understand the material: 

24. The computerized presentation helped me to better remember the material: 



ST DISAGREE : DISAGREE j UNDECIDED jAGRBE : ST. AGREE 
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02 pre . 0 
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04 control 5.3 
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paj iicularly effective in helping students understand and remember the material. It was not. 
On both questions the test groups were worse than the control (although not significantly 
soon question 24) and much worse than the pre-experiment classes (see Table 1), Table 3 
shows, in particular, the much greater frequency with which people in the test classes 
strongly disagreed with these two questions. This means either that the theory is incorrect 
or that the electronic blackboard was unable to capitalize on students' visual orientation. 
Our evidence points to the second conclusion. Some students did express a visual 
orientation, "Debbie*" commented, "Then again pictures are worth a thousand words and 
you remember those things in times of exams" while "Betty" added, "I think because when 
you are thinking of some of the key points you think, what was on the screen at the time 
and it helps you remember back to what they were talking about," Yet they and most 
others found the electronic blacldjoard actually interfered with understanding and 
remembering the material, "Elsie" was typical of most. When asked why she thought the 
electronic blackboard made things worse than a normal lecture, she responded: "I think so 
because you tended to concentrate, actually, on the graphics and not listen to what the 
professor was saying " As was the case with note-taking, distraction and divided attention 
negated whatever benefits the visual images of the electronic blackboard might have had. 
Two other questions show strong differences between the control and test classes. 
The chief complaint against the experiment was that it was boring (question 18: "The 
presentation was boring,"). That was the overwhelming opinion of students in the control 
group (see Table 4). Students in the test classes also found the mater al boring, although 
significantly less so than the control group, "Cindy" summed up the feeling of most 
students: "I am not all that interested .„ in this particular topic, no matter how it probably 
was presented," The control class proved that illustrated lectures without illustrations are 
tedious. The test classes proved pictures alone are not enough to enliven unpopular 
material. From the interviewees' responses, it seems that interest was the chief factor in 
the answers to questions 11, 12, 13, 14 and 16 measuring effectiveness, appropriateness, 
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TABLE 4 

FREQUENCY DISTRIBUTION PERCENTAGES 



18. The presentation was boring: 
19. The presentation was hard to see: 
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quality, and making comparisons. As one student in the morning class wrote: "I want to 
be entertained." 

Question 19 ("The presentation was hard to see.") is the other place where major 
differences appear between the control and test groups. Some students had trouble seeing 
the overhead slides used for graphs in the control group, but as Table 4 shows, many 
students had real trouble seeing the electronic blackboard. The night class saw the 
presentation through an RGB projector of admittedly mediocre quality. The large screen 
TV used for the day class had much better resolution but not enough to satisfy students. 
Since it has been our experience that weaker students tend to sit towards the back of the 
classroom, the difficulties people farther away from the screen had in seeing would tend to 
negate the benefits which these students might have gained from the clearer organization of 
the material on the electronic blackboard. 
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One difficulty in interpreting these results is created by the selectivity problem. To 
what extent are these classes fully comparable? On the pre-experiment evaluations, 
Knuttila's day class consistently gave higher scores than his night class (excepting only 
question 22, "I am afraid to interrupt."), although both classes received the same material 
in the same format (see Table 1 ). During the test, the day class was consistently less 
favourable than the night class. (The night class gave higher scores on only 3 of 1 7 
questions). There was also a fair amount of bimodalism in the response curves. The day 
class was bimodal on 8 of 17 questions {ffs 10, 12, 13, 18, 19, 20, 25 & 26); the night 
class on 3 questions (#s 19, 20 & 22); and the control group on 2 of 16 questions (#s 18 & 
19). While the day class was the most negative, it also was a group with divided opinions. 
Obviously, uncontrolled factors, such as the time of day, configuration of the classroom, 
age and maturity of the students, etc., account for some of the variance. 

To better analyze these factors, we created profiles of students who were 
particularly favourable or unfavourable to the electronic blackboard. We defined a student 
as favourable who answered 4 or 5 (accounting, of course, for those questions where the 
scale is inverted) on nine to eleven of seventeen questions and very favourable if on twelve 
or more. Unfavouiable and very unfavourable were defined as answering 1 or 2 on 
questions in the same proportion. The differences between the two classes stand out (see 
Figure V). 'Ilie night class had more students in the neutral category while in the day class 
those who were very unfavourable were three times as numerous. Unfortunately, 
determining what caused these differences proved elusive. The socio-economic and 
academic data we collected established few patterns. 

Surprisingly, gender did not appear to be a major factor (see Table 5). In the night 
class males averaged .05 less favourable than women, but in the day class the men 
averaged .17 more favourable (although four of the nine very unfavourable students in the 
day class were men, at 44.4% considerably greater than their proportion in the class). On 
some of the questions, strong differences between men and women in one class were 
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FIGURE V 

STUDENT PROFILE 
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cancelled by results from the other. The only consistent pattern seems to be that women 
rated the instructor more highly (question 17). 

Other socio-economic categories were similarly unenlightening. Only one person 
whose first language was not English fell into either profile. As Table 6 shows, there is 
little difference between the profiles in terms of their age, the number of classes taken 
before this semester, or their cumulative grade point average. 

TABLE 6 



STUDENT PROFILE 

MEAN OF SELECTED SOCIAL VARIABLES 
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On the other hand. Table 6 does appear to show some association with residence 
and income. Table 7 clarifies these associations. In terms of residence, urban students 
appear to be somewhat more favourable than rural students. Most of the variance comes 
from students from small cities who were favourable and those from farms who were not. 
In the case of income, it appears that students whose families' incomes are under $25,000 
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are considerably less favourable. There is a problem with these data, however. Four of 
the 32 students in the negative profile (12,5%) declined to answer the question, while only 
one positively profiled student declined (4%), Given the small numbers involved, this 
could considerably alter the results. Indeed, the small size of the sample gives u,s little 
confidence in the reliability of the data for either question. Our interviews gave us no 
additional insights here. If the electronic blackboard impacts more negatively upon low 
income and rural students it could be of great significance for the project's design model, 
but our evidence is too equivocal to draw that conclusion. 

Academic factors were much less equivocal (see Table S), Social science majors 
were much more likely to view the electronic blackboard favourably, "Other" is a residual 
category which consists mostly of students who have not yet declared a major. These 
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students make up nearly half of those in the negative profile. This should not be 
surprising. The content of the three lectures was very "sociological," and social science 
majors could be expected to appreciate it more. On the other hand, most of the students in 
an introductory sociology class at the University of Regina are taking ii only to fulfill 
university requirements. During the interviews several students said they thought the 
electronic blackboard would have been more appropriate in classes which depend upon 
equations and formulas, such as economics, mathematics, or engineering. If the electronic 
blackbf ai-d is unable to satisfy introductory students in sociology, it may be more effective 
in upper level classes or in other disciplines. 

The personality of the professor was another uncontrolled factor which seemed to 
affect the outcome. Many students made reference to some aspect of the professor's 
personality and style- Anderson's effective use of personal examples, Knuttila's humour, 
Stahl's knowledge of the subject. We specifically asked all of the interviewees if they 
thought that having their regular professor (Knuttila) deliver the presentations would have 
made a difference. Six of the ten thought it would, only two (both from the night class) 
unequivocally thought it would not. Many agreed with "Cindy:" "I am not used to his 
[Stahl's] teaching methods, so it is hard for me to make a good comparison or good 
evaluation of what I think." Teaching is a dynamic process. It is quite clear that content, 
method, format, and style are all a part of that process. Students become used to a 
particular style and personality, they have expectations about what will happen in class. A 
guest lecturer brings an element of novelty. Some look forward to it, as "Al" did: "I 
thought with a different prof you are looking for something different. You are looking 
forward to seeing him so you will be paying a little bit more attention because it is a 
different kind of style." But novelty can soon wear thin, and differences in style and 
format (particularly a very different format such as the electronic blackboard) can cause 
problems for some. As "Alice" put it: "It took the full three days to really get used to it." 
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We cannot quantify the extent of the human factor in the experiment, but the students insist 

it is integral to the process of teaching. 

Finally is the factor of student expectations, A number of student comments 

expressed disappointment in what they saw. "Elsie'' spoke for these when she said: 

If this computerized technology is going to be used for individual instruction I think 
a lot more has got to be done. Just programming and graphics and headlines and 
major points aren't necessarily the way to go; you can do that on an overhead 
screen, you can do that on xeroxed pieces of paper, I think I was expecting a lot 
more, I don't know why and I'm really not sure what I was looking for in the class 
itself, I just thought that the computerized section was sadly lacking something. 

Most students are familiar with the excitement of video games and the special effects of 

Lucas and Disney, They have all seen the advertising hype surrounding computers which 

so oversells their abilities. So while for us the point of departure for the electronic 

blackboard was the traditional chalkboard and overhead, many students may have expected 

Star Wars and Nintendo, What they got could not help but be disappointing by 

comparison. As with other human factors, their disappointment cannot be quantified but it 

certainly played some role in their response, 

CONCLUSIONS 

^rhe electronic blackboard was not successful in our experiment. We measured 
negative impacts on all of our variables. Faculty productivity declined sharply. We had 
expected lengthy preparation time, but not of the magnitude which we encountered. It took 
five times as long to prepare an illustrated lecture as a normal one for a moderately 
experienced user whose research was compkte and who had all materials at hand. In spite 
of this, teaching quality declined, in the estimation of both the lecturer and the faculty 
review panel, as the electronic presentation interfered with normal patterns of classroom 
interaction. The students were dissatisfied with the electronic blackboard. We deliberately 
chose unpopular material for the experiment, as is reflected in sharply negative responses 
in the control class, Tlie electronic blackboard did othing to improve this, Ihe 
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evaluations of both test classes were not significantly different from the control group but 
were much worse than the pre-experiment classes. 

The most successful aspect of the electronic blackboard was the organization of the 
presentation. The illustrated lectures were less boring than the same material presented 
without illustrations, but much more boring than the pre-experiment lectures. Contrary to 
expectations,^ the illustrations did not help students understand or remember the material 
better, nor to take notes better. There were several reasons for this. Many students found 
the presentation hard to see. More importantly, the electronic blackboard forced students to 
concentrate on two different sources of information simultaneously. The resulting division 
of attention and distraction more than negated whatever benefits might have been gained by 
superior organization and visual imagery. 

A number of uncontrolled factors influenced the outcome. These included the 
layout and lighting of the classrooms, the personality and style of the professors, and 
student expectations. Social science majors were more favourably inclined to the 
presentations, while those who had not declared a major were much less so. Gender was 
apparently not a major factor, but there is some (albeit not very reliable) evidence that rural 
and low income students were negatively affected by the electronic blackboard. 

Given the current state of the technology, the electronic blackboard is not effective 
under the conditions tested. It remains possible that in smaller, upper level classes or in 
more technically-oriented disciplines it may still prove useful. 

If the electronic blackboard is to have a place in the lecture hall, we believe that 
three obstacles will have to be overcome. The first is in display technology. Until 
affordable, high-quality display technology is readily available, the electronic blackboard is 
not suitable for large classes. The computer industry has done well in getting high- 
resolution colour graphics to the computer's monitor screen. The problem is in displaying 
those images to a large group. This is crucial. If students cannot see the presentation 
clearly, the electronic blackboard is worse than a waste of time and resources. Should a 
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university wish to pursue the electronic blackboard as a teaching device, considerable 
attention must be paid to acquiring and deploying suitable display technology. Unless that 
technology is easily mobile and effective under a variety of lighting conditions, dedicated 
classrooms will have to be constructed. Unless conversion from monitor to large screen is 
simple, technical assistance will have to be provided. Computers and software are only the 
initial expense. 

The second obstacle is preparation time. The time involved in producing illustrated 
lectures is prohibitive. Better, easier-to-use software and a wider selection of appropriate 
electronic clip art would be of some help, but to a large measure this problem is inherent in 
the electronic blackboard. Should a university wish to develop the electronic blackboard, it 
will have to seriously consider giving faculty release-time from normal duties for 
preparation. Once developed, of course, it should not be as time consuming to revise an 
illustrated lecture, but we suspect revision time will be considerable as well. 

The third obstacle is the divided attention and distraction which the electronic 
blackboard produced in the classroom. Most of the interviewees and many written 
comments dwelt on the importance of the interaction between professor and student and the 
rapport which grows from it. The key to quality teaching is in these human factors. To be 
successful, any electronic aid would have to enhance these factors. Our experience with 
the electronic blackboard was that it seriously interfered with them. This is an obstacle 
which, frankly, we do not see as amenable to technical improvement. Indeed, it raises 
questions which the evaluation of all forms of electronic teaching need to address. Unless 
this obstacle can be overcome, there is not much point in pursuing the electronic 
blackboard. 

These three obstacles are serious impediments. Some forms of computerized 
teaching may be effective in the university classroom, but we believe that at this time the 
electronic blackboard is not one of them. We do not recommend its incorporation into the 
design model. 
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AITKNDIX A 



CONSENT FORM 



ERIC 



ELECTRONIC BLACKBOARD EXPERIMENTS 
CONSENT FORM 



This class will involve electronic blackboard experiments 
conducted by Drs. Chandler, Knutilla, Maguire^ and Stahl . These 
experiments are part of a project to develop a design model for a 
computerizing university, funded by the Canadian International 
Development Agency and the government of India. The purpose of 
these experiments is to discover whether or not microcomputer- 
based presentational technology (electronic blackboards) improves 
faculty productivity, the quality of teaching, and the quality of 
learning . 



I understand that several times during the class the 
professor or a guest lecturer will make presentations of class 

material using computer technology and that I will be asked to 

evaluate these presentations. I understand that as part of the 

experiment I will be asked to evaluate non-computer portions of 

the class as well. These evaluations will involve me in filling 
out questionnaires and being interviewed. 

I consent to participating in these evaluations. I 
understand that questionnaires will be anonymous and that 
interviews will be kept confidential. I also understand that I 
am free not to participate if I so choose and that my marks for 
this class will not be affected, either way, by my choice. 



name (please print) signature 



student number date 
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APPKNDIX B 

B.1 PRE-EXPliRIMENT AND CONTROL QUESTIONNAIRE 
B.2 TEST QUESTIONNAIRE 
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ELECTRONIC BLACKBOARD EXPERIMENTS 

EVALUATION QUESTIONNAIRE 

DO NOT SIGN YOUR NAME OR STUDENT NUMBER TO THIS FORM. 
CLASS NAME AND NUMBER: . 



Please circle the appropriate number. 



1. In what year were you born? 19 

2. Sex female 1 

ma 1 e 2 

3. Faculty or Program Administration 01 

Arts (incl. Campion & Luther)... 02 

Education 03 

Engineering 04 

Grad. Studies and Research 05 

Physical Activity Studies 06 

Sask. Indian Fed. College 07 

Science (incl. Camp.& Luther).., 08 

Social Work/Human Justice 09 

University Entrance Program 10 

Other 11 

4. Major Sociology 01 

Other Social Science 02 

Computer Science 03 

Other Natural Science 04 

Pre-Administrat ion 05 

Humanities 06 

Fine Arts 07 

Other 08 

5. How many classes have you taken before this semester? 

0 1 

1-2 2 

3-4 3 

5-8 4 

more than 8 5 

6. What is your cumulative grade point average? 

less than 59% 1 

60-69% 2 

70-79% 3 

80-89% 4 

90% or better 5 
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7. Is English your first language? 

yes 1 

no 2 

8. Where is your family from? 

large city (Regina, etc . ) 1 

small city (Moose Jaw, etc) 2 

town 3 

farm 4 

reserve 5 

9. What is your family's annual income? 

under $14,999 ; 1 

$15,000 - $24,999 2 

$25,000 - $34,999 3 

$35,000 - $44,999 4 

$45,000 - $54,999 5 

$55 , 000 or more 6 

Please rate the class presentation by circling the number 
that represents the response closest to your opinion: 

l=poor 2==fair 3=average 4=good 5=excellent 

10. The organization of this 

presentation was: 12 3 4 5 

11. Compared with other portions of 

this class, this presentation was: 12 3 4 5 

12. The effectiveness of the 

presentation was: 12 3 4 5 

13. The appropriateness of the 
presentation's format for the subject 

matter was: 12 3.45 

14. Compared to other classes I have 

taken, this presentation was: 12 3 4 5 

15. The pace at which the presentation 

covered the material was: 12 3 4 5 

If you answered l(poor) or 2 (fair), 
was the presentation: too slow....l 

too fast .... 2 

16. I would rate the overall quality 

of the presentation as: 12 3 4 5 

17. Apart from the material presented, 

I would rate the instructor for this 

presentation as: 12 3 4 5 



Please rate the following statements by circling the number 
that represents the response closest to your opinion: 



l=strongly disagree 
4=agree 



2==disagree 3=undecided 
5=strongly agree 



18. The presentation was boring: 

19. The presentation was hard to see: 

20. The presentation helped me take 
notes better : 

21. This material is too difficult: 

22. During the presentation I am afraid 
to interrupt with a question: 

23. The presentat iori helped me to better 
understand the material: 

24. The presentation helped me to better 
remember the material: 

25. I would like to see more of this kind 
of presentation: 



2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 



ADDITIONAL COMMENTS 

A. Which aspects of the instructor's presentation did you 
appreciate most, and why ? 



B. Are there areas in which the instructor's presentation 
could be improved, and how ? 



C. Are there any other areas about which you would like to 
comment? 
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B.2 

ELECTRONIC BLACKBOARD EXPERIMENTS 

EVALUATION QUESTIONNAIRE 

DO NOT SIGN YOUR NAME OR STUDENT NUMBER TO THIS FORM. 
CLASS NAME AND NUMBER: 



Please circle the appropriate number. 



1. In what year were you born? 19 

2. Sex female 1 

ma 1 e 2 

3. Faculty or Program Administration 01 

Arts (incl. Campion & Luther)... 02 

Education 03 

Engineering 04 

Grad. Studies and Research 05 

Physical Activity Studies 06 

Sask. Indian Fed. College 07 

Science (incl. Camp.& Luther)... 08 

Social Work/Human Justice 09 

University Entrance Program 10 

Other 11 

4. Major Sociology 01 

Other Social Science 02 

Computer Science 03 

Other Natural Science 04 

Pre-Administration 05 

Humanities 06 

Fine Arts 07 

Other 08 

5. How many classes have you taken before thif? semester? 

0 1 

1-2 2 

3-4 3 

5-8 4 

more than 8 5 

6. What is your cumulative grade point average? 

less than 59% 1 

60-69% 2 

70-79% 3 

80-89% 4 

90% or better 5 
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7. Is English your first language? 

yes 1 

no 2 

8. Where is your famiW from? 

large city (Regina. etc.) 1 

small city (Moose Jaw, etc) 2 

town 3 

farm 4 

reserve 5 

9. What is your family's annual income? 

under $14,999 1 

$15,000 - $24,999 2 

$25,000 - $34.999 3 

$35,000 - $44.999 4 

$45,000 - $54,999 5 

$55 , 000 or more 6 

Please rate the class presentation by circling the number 
that represents the response closest to your opinion.- 

l*poor 2»=fair 3-average 4==good 5=excellent 

10. The organization of this 

presentation was: 12 3 4 5 

11. Compared with other portions of 

this class, this presentation was: 12 3 4 5 

12. The effectiveness of the 

presentation was: 12 3 4 5 

13. The appropriateness of the 
presentation's format for the subject 

matter was: 12 3 4 5 

14. Compared to other classes I have 

taken, this presentation was: 12 3 4 5 

15. The pace at which the presentation 

covered the material was: 12 3 4 5 
If you answered Kpoor) or 2 (fair), 

was the presentation: too slow....l 

too fast .... 2 

16. I would rate the overall quality 

of the presentation as: 12 3 4 5 

17. Apart from the material presented, 

I would rate the instructor for this 

presentation as: 12 3 4 5 



Please rate the following statements by circling the number 
that represents the response closest to your opinion: 



l^strongly disagree 
4=agree 



2=disagree 3=undeclded 
5=strongly agree 



18. The presentation was boring: 

19. The presentation was hard to see: 

20. The presentation helped me take 
notes better: 

21. This material is too difficult: 

22. During the computerized presentation I 
am afraid to interrupt with a question: 

23. The computerized presentation helped me 
to better understand the material: 

24. The computerized presentation helped me 
to better remember the material : 

25. I would like to see more of this kind 
of presentation: 

26. I find the computerized presentation 
distracting : 



2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 



ADDITIONAL COMMENTS 

A. Which aspects of the instructor's presentation did you 
appreciate most, and why ? 



B. Are there areas in which the instructor's presentation 
could be improved, and how ? 



C. Are there any other areas about which you would like to 
comment? 




